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The primary objective of the project for Improving 
the Teaching of English as a Second Language in the High Schools was 
to improve the facility of English-language-handicapped students in 
the four language skills (listening, speaking, reading, and writing) 
so that they can make a proper adjustment to high school. Host of the 
students are from Spanish-speaking or French-speaking backgrounds, 
althought the project serves as many as 20 different language groups. 
To overcome the language problem of these students in the 50 high 
schools selected for participation, three (later increased to four) 
teacher trainers were employed at the Central Board, a variety of 
curriculum materials was uniformly introduced to the schools, and a 
structure was provided to coordinate and direct the overall effort. 
Specifically, the program objectives which were evaluated were: (1) 
the efficacy of the teacher-training program; (2) the number of 
classroom visitations, workshops and demonstration lessons offered by 
the Central Board £taff of teacher-trainer consultants; (3) the 
degree to which the project has made teacher participants more aware 
of the special needs of non-English speaking learners; (4) the amount 
of articulation among high schools in the project reported by ESL 
department chairmen; (5) a comparison of the dropout rate of ESL 
students in the program with comparable grade level and aged children 
in conventional classes; and, (6) student growth on the Puerto Rican 
Scale "A", a rating scale used by classroom teachers to evaluate the 
children's ability to speak and understand spoken English* Thu 
Linguistic Capacity Index was also used to assess pupil achievement 
in. learning English as a second language. (Author/JM) 
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EXECUTIVE SUMMARY 



The ev^kmors d\d not receive a contract for the evaluation until late in February, 1973, at 
which time it wab decided to use as much of the research design prepared by the Bureau of 
Educational Research of the Board of Education of the City of New York (August 1 972) as was . 
feasible considering the time limitations imposed by the late start, with specific reference to pre- 
and post- testing as well as the preparation of instruments needed for evaluating some of the 
program objectives. Other information was not available when requested; this included data on 
students who had left the ESL program the previous year and were currently in the mainstream 
as well as data from the heads of guidance departments in the feeder junior high schools. The 
evaluators were precluded by time from preparing achievement tests in reading and in writing for 
different proficiency levels, and tests to determine whether students could read and write func- 
tionally to enter the mainstream; nor was the Metropolitan Reading Test administered in April 
1973. As a result, all program objectives reflecting the need for these data have been omitted 
from the report. 

Program Objectives 

The program objectives which were evaluated were: 

1 . The efficacy of the teacher-training program 

2. The number of classroom visitations, workshops,'and demonstration lessons offered 
by the Board staff of teacher-trainer consultants 

3. The degree to which the project has made teacher participants more aware of the 
special needs of ESL learners 

4. The amount of articulation among high schools in the project reported by ESL 
department chairmen 

5. A comparison of the drop out rate of ESL students in the program with comparable 
grade levels and aged children in conventional classes, and 

6. Student growth on the Puerto Rican Scale "A". 



Findings 

The failure of the data to entirely support attainment of any of the evaluation objectives 
described herein should be considered more of a failure in proposing realistic objectives than of a 
failure of the program or its personnel 

Student Selection and Placement. Eighty-nine percent of the teachers stated that student 
placement was based on English language proficiency; yet, 69% indicated that such proficiency 
was determined by an interview alone without reference to the Puerto Rican Scale A (also referred 
to as ScaJe) or the Linguistic Capacity Index (LCI), indicating that student placement is done very 
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subjectively. This notion was supported by teacher responses that only 14% indicated that all of 
their classes were homogeneous, and also by examination of the LCI pre-test scores which overlap 
each of the Scale categories from F to C. 

The Teaching Staff. More than 50% of the teachers in the program have six or more years of 
teaching experience while less than 20% have six or more years of ESL teaching experience. More 
than 67% have the fundamental professional courses required for ESL teachers, but less than 40% 
are members of the national professional organization (TESOL) and only -15% are members of the 
state affiliate (NYSESOLBEA). These percentages are consistently lower with the less-experienced 
teachers. Almost 60% of the program ESL teachers have City ESL licenses. 

In-Service Training. The stated objective that 90% of the teachers would rate the consultants 
good to excellent was not achieved for any of the items prepared by the evaluation team. Never- 
theless, teacher response was consistently favorable with the exception of their responses to both 
workshops and demonstration lessons, which are discussed further in the next finding. 

Numbers of Visitations, Workshops and Demonstration Lessons. The stated objective of a 
monthly average of 16 visitations, 1 workshop and 5 demonstration lessons was achieved only by 
the average number of visitations (19.54), The terms workshop and demonstration lesson were 
nowhere defined in the proposal and it was the impression of the evaluators that both teachers 
and the consultants themselves were unclear as to what constituted a workshop and what a demon- 
stration lesson. This was further complicated by the evaluation director's understanding that the 
workshops were intended specifically for the department chairmen and not for the classroom 
teachers. Consequently, the evaluators recommend ignoring all data concerned with workshops 
and demonstration lessons as invalid. The important consideration is that the teacher-trainer 
consultants were out in the schools where they were supposed to be, providing services for which 
they were employed. 

Awareness of the Special Needs of the ESL Learner. Although teacher response was on the 
whole positive, no item received the 90% affirmative response required to achieve the stated 
objective that the program had made them more aware of the ESL learner's special needs. It was 
unwise to assume that 90% of any professional group would admit that they had that much to 
learn about their own field. A more realistic objective of 60% would have permitted nine of the 
fifteen items surveyed to have attained that objective. 

Articulation. Only 35% of the department chairmen surveyed believed that there' was articu- 
lation between their high schools and the feeder junior high schools, failing to achieve the 90% 
affirmative response in the stated objective, mitigated by the statements by many of the chairmen 
that most of their students are new arrivals. According to the Assistant Director of the program, 
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district coordinators attended institutes held by the Office of Bilingual Education, as well as per- 
sonnel from the junior high schools who spoke to the high school representatives about articulation. 
The program Assistant Director also stated that the teacher trainers assisted in articulation in their 
visits to their schools. 

The efforts of the Central Board received strong praise from the chairmen, 74% of whom be- 
lieved that there was articulation in terms of similarity of ESL programs in the project high schools. 
This was supported by 83% of the teachers who indicated that they met their classes 5 to 10 periods 
weekly while 91% were using at least one of the four types of classroom materials distributed by the 
Board. 

Effects of Program on Students. 

1. Drop Out Rate 

The combined average ESL drop out rate was 50% better than the combined total 
dropout rate for the limited sample of responding schools, thereby meeting the stated objective. 

2. Puerto Rican Scale A 

The stated objective of attaining at least one level higher on Scale A (speaking ability) 
was reached by the appropriately projected percentages for students who pre-tested into levels 
D and E. Of those who pre-tested in at level F, 86% were rated at least one level higher but this 
figure failed to reach the projected figure of 100%. Only 59% of those who pre-tested at level C 
attained level B or higher failing to reach the projected figure of 75%. Information obtained by 
using the Linguistic Capacity Index suggests that the major fault of any failure to reach the 
stated objectives on the Puerto Rican Scale was probably caused by poor initial placement. (See 
comparison below, #4.) 

i. The Linguistic Capacity Index 

Although this test seems better suited to elementary school youngsters, it was used for 
two reasons: pre-test data were available when the evaluators received the contract in late February, 
and it would provide a comparable measure to the Scale ratings. (A separate examination of the 
LCI by the evaluation team can be found in Appendix N.) The t-ratios for the pre-post test mean 
gains were significant at the .005 level for each of the individual components of the test and for 
the test as a whole. 

4. Puerto Rican Scale A and the Linguistic Capacity Index 

The LCI scores were set up in groups according to those students rated in each of the 
categories (C to F) on the Scale. What was immediately clear was that despite ascending means on 
the LCI for each group from F through C, there was considerable overlap among scores on the 
LCI for students at each of the Scale levels. This suggests that the program did not have homogen- 
eously grouped classes for the most part because of a dependence on a rating. system that does not 
permit uniform ratings because it is too reliant on teacher judgment. 

For two presumably comparable instruments, the correlation coefficient (r) was a rather 
low .5230. 
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Recommendations 

In view of the evaluators' belief that any failure was more in the proposing of realistic 
objectives than in the program itself, the evaluators recommend that the program be continued. 
They further recommend: 

1. that all terms be clearly defined in any subsequent proposals, including awareness, 
articulation, workshop, demonstration lesson. 

2. that more realistic statistical goals be stated for future proposals, taking into account 
the diversity of human populations involved: students, teachers, department chairmen, and 
teacher trainers. 

3. that new instruments be sought or created for the purpose of student selection and 
homogeneous placement. 

4. that unappointed teachers licensed by the Board of Examiners to teach ESL be hired 
whenever there is an opening in the program. There is no reason that more than 40% of the 
teachers do not have ESL licenses. At the same time, the Board should be credited for attempting 
to provide in-service training for those who do not hold licenses and even for those who do. 

5. that, if workshops and demonstration lessons are to be included in subsequent pro- 
posals, they be scheduled in school and at the Board at times convenient for the maximum 
number of personnel to benefit from them. 

6. that, if articulation between the feeder junior high schools and the high schools is 
solely a function of these schools, then it should not be listed as a function of the Central Board 
and should thus be omitted from program objectives. 

7. that the efforts to disseminate throughout the schools information about, and newly 
published materials in, the field of English as a second language be continued and intensified. 

8. that, if information on students who leave the program for the mainstream is sought for 
follow-up purposes, procedures should be'established so that such information is readily available. 

9. that, if follow-ups are to be done on drop out rates, once again procedures must be 
established, so that such data are readily available, both for current year students and for those 
students who might have moved into the mainstream after having been "promoted" from the 
program in the current or previous years. 

10. that instruments be created for testing student achievement in all language areas and 
skills taught in the program. 

1 1 . that the Central Board staff foster interest in and encourage participation in appropriate 
professional organizations at the national (TESOL) and local levels (NYSESOLBEA). 

12. that consideration be given to providing at least two periods of ESL daily (this is 
already being done in some schools), especially to those youngsters whose English proficiency is 
low. A regular English class can do little but frustrate such youngsters, who would benefit 
considerably by the double period or two separate periods of ESL. 
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INTRODUCTION 



The evaluation team did not receive the contract for the evaluation until !ate in February, 
1973, at which time it was decided to use as much of the research design prepared by the Bureau 
of Educational Research of the Board of Education of the City of New York (August 1972) as was 
feasible considering the time limitations imposed by the late start, with specific reference to pre- 
and post-testing as well as the preparation of instruments needed for evaluating some of the 
program objectives. Other information was not available when requested; this included data on 
students who had left the ESL program the previous year and were currently in the mainstream 
(attendance records, drop out rates, academic achievement, school behavior, and ratings on the 
Puerto Rican Scale U A"), as- well as data from the heads of guidance departments in the feeder 
junior high schools. The evaluators were precluded by time from preparing achievement tests 
in reading and in writing for different proficiency levels and tests to determine whether students 
could read and write functionally to enter the mainstream; nor was the Metropolitan Reading 
Test administered in April 1973. As a result, all program objectives reflecting the need for these 
data have been omitted from the report. 

PROGRAM OBJECTIVES 

The primary objective of the project for Improving the Teaching of English as a Second 
Language in the High Schools was to improve the facility of English-language-handicapped 
students (defined as those rated "C" or below on the Puerto Rican Scale "A") in the four 
language skills (listening, speaking, reading and writing) so that they can make a proper adjust- 
ment to high school. Most of the students are from Spanish-speaking or French-speaking back- 
grounds, although the project serves as many as twenty different language groups. To overcome 
the language problem of these students in the fifty high schools selected for participation, three 
(later increased to four) teacher trainers were employed at the Central Board, a variety of 
curriculum materials was uniformly introduced to the schools, and a structure was provided to 
coordinate and direct the overall effort. 

Specifically, the program objectives which were evaluated were: 

1. The efficacy of the teacher-training program 

2. The number of classroom visitations, workshops and demonstration lessons offered 
by the Central Board staff of teacher-trainer consultants 

3. The degree to which the project has made teacher participants more aware of the 
special needs of non-English speaking learners 

4. The amount of articulation among high schools in the project reported by ESL 
department chairmen 

5. A comparison of the drop out rate of ESL students in the program with comparable 
grade level and aged children in conventional classes, and 

6. Student growth on the Puerto Rican Scale "A". 



EVALUATION OBJECTIVES 



The evaluation of the program assessed the degree to which the program objectives were 
met. Consonant with the program objectives, the evaluation objectives were: 

1 . The teacher-training program will be rated good to excellent by 90% of the teachers. 

2. Records will indicate that consultants from the Board office will have averaged 
sixteen visits, two workshops and five demonstration lessons per month. 

3. Ninety per cent of the teacher participants will indicate that the project has made 
them more aware of the special needs of non-English-speaking children. 

4. Greater articulation among high schools will have been reported by 90% of the 
chairmen surveyed. 

5. The drop* out rate of ESL students in the program will be 50% less than that of 
comparable grade level and aged children in the conventional classes. 

6. At the end of the school year, pre-post-test student ratings by teachers on the Puerto 
Rican Scale "A" will indicate the following improvement in speaking ability: 

1 00% of those originally rated "F" will be rated "E" or better. 
80% of those originally rated "E" will be rated "D" or better. 
60% of those originally rated "D" will be rated "C" or better. 
75% of those originally rated "C" will be rated "B" or better. 

METHODS OF COLLECTING DATA 

Evaluation of teachers' program rating and teachers' growth in awareness was made through the 
use of the "Questionnaire for Teachers" (See Appendix A). 

Evaluation of the consultants' activities was made through the use of the "Account Form 
for Teacher-Trainer Activities" developed by the Bureau of Educational Research (See Appendix B). 

Evaluation of the articulation among high schools was made through the use of the "Articu- 
lation Survey" form (See Appendix C). 

Evaluation of the dropout rate of ESL student project classes compared to the school-wide 
dropout rate was made through the use of the Drop Out Data Form (See Appendix D). 

Pupil progress in language proficiency was determined in two ways. Classroom teachers used 
the Puerto Rican Scale, Form A, to evaluate the children's ability to speak and understand spoken 
English on a pre-and post-test basis. The Linguistic Capacity Index (Brengelman, Frederick H. and 
John C. Manning. 1964. Southwest Educational Development Laboratory, Austin, Texas) was 
also used to assess pupil achievement in learning English as a second language. 
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THE PROGRAM IN OPERATION 



This re-cycled program was expanded to include over fifty high schools in Manhattan, the 
Bronx, Brooklyn and Queens, servicing non-English speaking students rated "C" to "F" on the 
Puerto Rican Scale A, who were referred by classroom teachers and guidance counsellors in the 
respective schools. In addition, teacher-training services were performed by four (originally three, 
increased in February) teacher-trainer consultants each of whom was assigned to specific high 
schools and was involved in classroom visitations, workshops, demonstrations lessons and the 
dissemination of new materials and media for the program. 

The program was established to increase pupil performance in the four basic language skills 
(listening, speaking, reading, and writing) and to train teachers and supervisors in the techniques 
of teaching English as a second language. The program funded positions for high school teachers 
of English as a second language who were to devote five periods daily to direct instruction. Pupils 
were to attend a daily period of ESL instruction in addition to the regular English class. 

Selection of Student Population 

Students were to be selected for the ESL program on the basis of having been referred by 
the guidance counsellor and/or a classroom teacher and of having been rated "C" to "F" on the 
New York City Scale of Pupils' Ability to Speak English (Puerto Rican Scale A). In response to 
item 49a on the "Questionnaire for Teachers" (Appendix A), 89% of the teachers indicated that 
students were in fact placed in their ESL classes according to their English language proficiency 
alone, while the remaining 1 1% indicated that previous schooling and/or age of student were also 
taken into account (See Table I). 

Table 1 

ESL STUDENT PLACEMENT 

How are Students Placed in your ESL Classes? 
N = 69 

Number of Teachers Percent 

English Language Proficiency 61 89 

English Language Proficiency and 

Previous Schooling 5 7 

English Language Proficiency, Previous 

Schooling and Age of Student 3 4 

However, in response to item 49b on the same "Questionnaire," only 1% of the teachers . 
indicated that the Scale was in fact used for determining English language proficiency, another \% 
said that the Linguistic Capacity Index test was used for that purpose, but 69% of the teachers 
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indicated that interviews were used to determine English language proficiency, suggesting that 
any placement on an English proficiency basis would be based on subjective evaluations. The 
remainder responded that they used some combination of an interview with the Scale and/or the 
Linguistic Capacity Index, or the Scale and the Linguistic Capacity Index (See Table 2), Accord- 
ing to the Assistant Director of the program, all chairmen and teachers were alerted to the fact 
that all pupils would be rated according to the scale in October and again in May or early June. 

Table 2 

DETERMINATION OF ENGLISH LANGUAGE PROFICIENCY 

How is English Language Proficiency Determined? 



N = 69 

Number of Teachers Percent 

Puerto Rican Scale 1 ] 

LCI. 1 j 

Interviews 47 69 

P.R. Scale and Interviews 5 7 

P.R. Scale and LCI. 2 3 

L.C.I, and Interviews 6 9 

P.R. Scale, L.C.I, and Interviews 7 10 



This h not to suggest that either the Scale or the LCI is a better-defined instrument for 
student placement than an interview - it merely indicates that some kind cf uniformly adminis- 
tered and evaluated placement procedure is needed. 

The subjectivity involved in the placement procedure was reflected in the mixture of both 
homogeneous and heterogeneous class groupings seen in the teacher responses to item 50 of the 
"Questionnaire" in Table 3. 



Table 3 

CLASS TYPES ACCORDING TO ENGLISH LANGUAGE PROFICIENCY 



What kind of classes do you have according to their 
English language proficiency? 
N -69 



All homogeneous 

Some homogeneous / some 

heterogeneous 
All hetergeneous 
No response 



Mumber of Teachers % 

25 36% 

31 45% 

10 14% 
3 5% 
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In addition, examination of the pre-test scores on the Scale and the LCI raise questions as 
to their suitability for use as instruments for placement purposes, which is discussed subsequently 
in the report. 

The Teaching Staff 

The teachers employed in the ESL program presented a wide spectrum of background, 
training and experience for the job, according to their responses to items 40 and 41 on the 
"Questionnaire." More than 50% of the teachers in the program have been teaching for 
six or more years, while less than 20% have six or more years of experience teaching ESL. 
Less than 1 5% oT the teachers have less than two years teaching experience, while more than 40% 
have less than two years of ESL teaching experience. (See Table 4.) 

Table 4 

CLASSROOM EXPERIENCE OF ESL TEACHERS 

ESL Experience To tal Experience 







Less than 


Two to five 


Six or more 




Total 


two years 


years 


years 


Less than two years 


28 41% 


9 


13% 


9 


13% 


10 14% 


Two to five years 


28 41% 


0 




1K 

i <-J 


22% 


13 19% 


Six or more years 


13 19% 


0 




0 




13 19% 


Total ESL experience 


69 100% 


9 


13% 


24 


35% 


36 52% 



More than two thirds of the teachers responding to item 52 on the "Questionnaire" have 
taken the basic professional courses required for ESL teachers in the areas of general linguistics, 
structure of the English language, English phonology, and ESL methodology, including theory 
and/or materials, as can be seen in Table 5. 

Responses to items 53 through 56 on the "Questionnaire" indicate that less than 40% are 
members of the national professional organization (TESOL: Teachers of English to Speakers of 
Other Languages) and less than 15% are members of the state local chapter (NYSESOLBEA: New 
York State English to Speakers of Other Languages and Bilingual Educators' Association), although 
almost 25% have participated in the local meetings. (See Table 5 ~ a table listing the frequency of 
responses appears as Appendix E.) Since the percentages are consistently lower with the less- 
experienced teachers, the area of professional development and awareness is one that should be 
focused on if the program is re-cycled in subsequent years. 

Item 39 on the "Questionnaire" asked teachers to indicate which New York City licenses 
they hold. The teachers were asked to check three categories: 1. TESL; 2. Speech, English, and 
Foreign Language; and 3. Other licenses. Almost 60% of the teachers in the program hold ESL 
licenses; less than 20% hold only ESL licenses with another 33% holding an additional license or 
licenses in speech, English and/or a foreign language. See Table 6. 
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Table 5 

PROFESSIONAL TRAINJNG AND INVOLVEMENT QUESTIONNAIRE 

Comparison of Teachers in Terms of ESL Experience 
Responses Expressed as Percentages 
N = 69 



TABLES 








(28) 




(28) 




(13) 








Teachers with less Teachers with 


Teachers with 


P/ease indicate whether ycu have had 






than two years 


two-five years 


six or more years 


thp foJIowinn roursps at an accredited 

\ll\r 1 \Jl l\J WW II l*f bvUl tfWU Ok Oil ObW 


Total 


Experience 


Experience 


Experience 


college or university. 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 


a. ESL methodology 


70 


30 


57 


43 


79 


21 


77 


23 


b. ESL theory and/or materials 


59 


41 


43 


47 


71 


29 


69 


31 


c. Phonology and/or phonetics 


81 


19 


82 


18 


79 


21 


85 


15 


d. Constrastive linguistics 


41 


59 


36 


64 


43 


57 


46 


54 


e. General linguistics 


84 


16 


86 


14 


86 


14 


77 


23 


f. Structure of English 


67 


33 


64 


36 


71 


29 


62 


38 


g. Transformational Generative 


















Grammar 


43 


57 


39 


61 


54 


46 


31 


69 


a member of National TESOL? 


36 


64 


14 


86 


43 


57 


69 


31 


a member of N.Y.S. ESOL BEA? 


14 


86 


11 


89 


14 


86 


23 


77 


Participation in TESOL 


















Conventions 


35 


65 


14 


86 


36 


64 


77 


23 


Participation in N.Y.S. ESOL 


















BEA Conventions 


23 


77 


15 


86 


25 


75 


39 


62 



Table 6 

N.Y.C. LICENSES HELD BY ESL TEACHERS* 

N = 69 

Years Teaching ESL 





Total 


Less than 2 


2 to 5 


6 or more 


TESOL 


41 


14 


17 


10 


Speech, English & Foreign Language 


55 


23 


23 


9 


Other 


6 


1 


3 


2 


TESOL 


13 


5 


5 


3 


Speech, English & Foreign Language 


27 


13 


11 


3 


TESOL, Speech, English & 










Foreign Language 


23 


9 


9 


5 


TESOL & other 


1 






1 


Speech, English & other 


1 


1 






TESOL, Speech, English & Foreign 










Language & other 


4 




3 


1 


TOTAL TEACHERS 


69 


28 


28 


13 



•Several teachers hold more than one license. 
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Teacher-Trainer Consultants and In-Service Training 

In order to survey the teachers' responses to the teacher-training provided by the Central 
Board, a five-scale questionnaire (Items 1 through 1 7 in the "Questionnaire for Teachers," 
Appendix A) was prepared by the evaluation team, with questions relating to the knowledge- 
ability of the consultants; the number, quality and sequencing of the workshops and demonstration 
lessons; and the type, availability and quality of the ESL textbooks, instructional materials 
and audio-visual aids provided by the Board. Responses to the survey can be seen in Table 7, 
which indicates the percentage of teachers rating the teacher-trainers in each of the five levels of 
the scale. Since the objective was that 90% of the teachers would rate the training as "good to 
excellent" Table 8 indicates the responses of teachers in terms of "at least 90%" being rated 
good to excellent and "below 90%" being rated less than good. Table 9 provides the same data in 
terms of the frequency of teacher responses to each of the items. Each of the tables was broken 
down into responses of teachers with varying years of experience teaching ESL in the hope that 
trends might appear in terms of how experienced versus less-experienced teachers responded to the 
questions and rated the training provided. No trends were apparent. Tables parallel to 7,8, and 9 
but broken down in terms of total teaching rather than total ESL teaching may be found in the 
Appendix: F, G, and H. 

The objective was not met for any of the items surveyed. (An alpha coefficient was calculated 
to measure the reliability among the items on the scale in terms of overlapping variance. The 
alpha was .90 and would indicate that these seventeen questions contained a high degree of 
reliability. Cronbach, L. J. "Coefficient Alpha and the Internal Structure of Tests." Psychometrika, 
1951, 16, 297-334.) 

In order to determine whether the program objectives concerning the number of visits, 
workshops and demonstration lessons were met, the "Account Form for Teacher Training 
Activities" developed by the Bureau of Educational Research (Appendix B) was used. In addition, 
items 20 through 23 on the "Questionnaire for Teachers" gave concomitant information on the 
same items. Before discussing the actual responses, it must be pointed out that the evaluators 
received the impression that the lines of delimitation among "visitations, workshops, and demon- 
stration lessons" were not clearly marked. Nowhere in the proposal or the research design does 
there appear a definition of each of these terms. The evaluators have the impression that both the 
Board personnel (teacher-trainers) and the participating teachers were not clear as to what con- 
stituted a visit, a workshop, or a demonstration lesson, and, therefore, both had difficulty 
responding to the items concerning these activities, as can also be seen in the teachers' ratings of 
both the demonstration lessons and the workshops. The Evaluation Director had the distinct 
impression that the participating teachers were not invited to the workshops at the Board which 
were held on Friday mornings, but that these activities were scheduled specifically for the depart- 
ment chairmen or their representatives (usually a classroom teacher) when they could not attend. 
Nevertheless, since these distinctions were never made in the proposal or the design, the data 
reflect the responses of the teachers (not the chairmen) to all questions regarding workshops and 
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demonstration lessons. Attention should be re-directed to Tables 7,8, and 9 at this time to note 
how far from the stated objectives teachers' reactions were to eat r i of the items concerned with 
demonstration lessons and workshops. The evaluators would suggest that the data be ignored as 
invalid because of the lack of specificity and definition. 

If the monthly averages of visitations, workshops and demonstration lessons were totaled, 
it can be seen that they exceed the total indicated in the program objectives, 16, 2, and 5 res- 
pectively. (See Tables 10 and 1 1.) This clearly indicates that the teacher-trainers were where they 
were expected to be: in the schools. The critical fact remains that they were in the schools where 
they were available to provide a variety of services. 

Table 11 

ACTUAL CONSULTANTS' ACTIVITIES vs OBJECTIVE 

Demonstration 



Visits Workshops Lessons 
Objective 16 2 5 

Average of ESL 

Consultants 19.54 5.05 2.95 



Table 12 

TEACHER SURVEY: 
WORKSHOPS PROVIDED IN SCHOOL AND ATTENDED 

How Many Workshops Were Provided in Your School? 



Score Frequency % X = .55 SUM = 38 

0 58 84 

1 3 4 

2 4 6 

3 1 1 
6 1 1 
9 2 3 

How Many Workshops Did You Attend? 

Score Frequency % X= 1.000 SUM = 69 

0 48 70 

1 6 9 

2 7 10 

3 2 3 

4 1 1 
6 2 3 
9 3 4 
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Table 13 . 

TEACHER SURVEY: DEMONSTRATION 
LESSONS PROVIDED IN SCHOOL AND ATTENDED 

How Many Demonstration Lessons Were Provided in Your School? 

SUM = 84 



Score 


Frequency 


% X= 1.2174 


n 
\j 


38 


55 


1 


9 


13 


2 


9 


13 


3 


4 


6 


4 


1 


1 


5 


7 


10 


6 


1 


1 




How Many Demonstration Lessons Did You Attend? 


Score 


Frequency 


% X= 1.2899 


0 


37 


54 


1 


10 


14 


2 


9 


13 


3 


5 


7 


4 


2 


3 


5 


4 


6 


9 


2 


3 



SUM = 89 



Teacher responses to items 20 through 23 regarding workshops and demonstration lessons 
can be seen in Tables 1 2 and 13. It should be pointed out that of all the teachers in the survey, only 
one teacher wrote a note stating that the lack of definition of terms made it difficult to respond to 
several items on the "Questionnaire," including those items that referred to workshops and demon- 
stration lessons. 

Awareness of ESL Student Special Needs 

Items 24 through 38 on the "Questionnaire for Teachers" were designed to determine 
whether or not the project had made the teachers more aware of the special needs of the non- 
English-speaking learners. Since the items which comprise teacher "awareness" are varied and 
subject to interpretation, rather than attempt to prepare a five-scale response, the evaluation team 
prepared a series of Yes/No questions concerning whether or not the program had increased 
teacher awareness of such areas as cultural differences, the nature of language, contrastive lin- 
guistics, the implications of linguistics and psychology for the ESL teacher, new materials and 
new approaches for teaching a variety of skills and subject areas to non-native speakers of English 
(Appendix A). 

Although teacher response to the kinds of items being tested were on the whole positive 
(See Table 14), at least 50% answered .yes to all but one of the questions, no item received the 
90% affirmative response required to meet the stated program objectives. Taking into account the 
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diverse backgrounds and experience these teachers bring to the program, it was unwise to assume 
that 90% of any such group of teachers would claim that they have that much to learn about the 
field in which they already have attained professional standing. No clear distinction emerges, 
however, to differentiate among the responses of teachers with less than two, two to five, and 
six or more years of ESL teaching experience as can be seen in Table 14.* 

Articulation 

The definitions of articulation assumed for this study were two-fold: I. coordination between 
the high schools and their respective feeder junior high schools, and 2. similarity among the ESL 
programs in the high schools participating in the project A two-part questionnaire (Appendix C) 
was sent to each of the department chairmen in the project to determine whether or not they 
believed there was articulation in both of the areas indicated above. Table 15 shows that only 35% 
believed that there was articulation with the feeder junior high schools, while 74% believed that 
the various high school programs were similar, both failing to meet the 90% criterion stated in 
the project objectives. 



Table 15 

CHAIRMEN RESPONSES TO ARTICULATION SURVEY 



Question 1— As regards the ESL Learners, 
is there successful coordination between your program 
and the junior high school from which they have come? 



Responses 

Yes 
No 

Don't know 
TOTAL 



Number 



11 
15 
5 
31 



Percentage 



35 
48 
17 

100 



Question 2— is there similarity between your ESL 
program and the ESL program of the other 
high schools participating in the project? 



Responses 

Yes 
No 

Don't know 
TOTAL 



Number 



23 
1 
7 

31 



Percentage 



74 
3 

23 
100 



* These data are presented in term- of frequencies and percentages of total teaching experience in Appendices J and K, 
respectively. 
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Chairmen were encouraged to add comments to their check-off responses. Many of those who 
responded negatively to the question concerning articulation with feeder junior high schools 
stated that most of their students are new arrivals who did not attend junior high schools in the 
City; others suggested that all efforts at coordination were initiated from the senior high schools' 
attempting to establish it through the guidance counsellor or the ESL coordinator at those schools 
with sufficiently large programs. 

Regarding the similarity among the high school programs in the project, several chairmen 
offered strong praise to the efforts of the Central Board staff for providing the leadership that has 
helped establish such articulation. Other chairmen credited their programs to the unique efforts 
of the teachers at their schools, who have prepared materials specifically designed for their own 
programs. Furthermore, these materials have been disseminated widely throughout the City high 
schools through direct distributions at the workshops held at the Board, with suggestions made i 
as to how they might be adapted by different schools to meet the needs of their programs. One ; . 
chairman suggested that though there is articulation in the ESL program, the same articulation ^ 
does not exist in other subject areas, implying that the concept of articulation should be extended 
to other subject areas with the kind of internally-controlled materials that .the non-native-English- 
speaking learner requires. 

Items 42 and 46 on the "Questionnaire for Teachers" asked questions concerning two areas 
in which articulation within the high schools could be determined. Item 42 dealt with the number 
of periods the teachers met with each ESL class they taught. As can be seen in Table 16, 83% of 
the teachers met their classes between five and ten times per week and only 9% of the teachers 
were not using any of the four types of instructional materials distributed by the Board. 

Table 16 

TEACHER SURVEY: 
ARTICULATION— CONTACT PERIODS AND TEXTBOOKS 

Question 42 Less than Five Five to ten More than ten 

f 70 2 58 10 

% 3 83 14 

One teacher sees different classes. 
5- 1 0 and more from 1 0. 



Question 46 


f 


" % 


New Lado Series 


56 


81 


American Book Company Charts 


34 


49 


English 900 


18 


26 


English for Today 


19 


28 


None of the Above 


6 


9 



Both teachers and chairmen were surveyed as to how they would rate the opportunity for 
input to the Board. The results can be seen in Table 17. 
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Effects of Program on Students 

Drop Out Rate. In order to determine the drop out rate for ESL students in comparison with 
those students in conventional classes, the evaluation team prepared a "Drop Out Data" form (See 
Appendix K), which was given to the ESL department chairmen in selected high schools. Time 
was a critical factor once again as it prevented the evaluators from getting the form sent out until 
late in May, which, in turn, resulted in only a small number of schools responding to the ques- 
tionnaire. Records were not directly available from the schools and it was only through the 
extraordinaiy efforts of the department chairmen that the evaluators were able to get the data 
presented here. However, in the sample of seven responding schools which can be seen in Table 18, 
the combined average ESL dropout rate was, in fact, 50% better than £he combined total dropout 
rate for the responding schools, which extend over the four boroughs and offer a wide enough 
sample to suggest statistical validity; nevertheless, the evaluators are aware that the sample of 
respondents was too small to make any general conclusion. 

The Puerto Rican Scale "A " Scale "A" is used by a teacher to rate the student's ability to speak 
English. The scale was administered on a Pre- and Post- basis in September and May. The data 
reported herein are limited to a sample of those students for whom the evaluation team was also 
able to get data on the Linguistic Capacity Index (LCI) for comparative purposes. The investiga- 
tors believed it would be useful to have correlative data on the two instruments, both of which 
have been used by the Board of Education for many years. (The results of the comparisons made 
between the two instruments are discussed in a later section.) 

The stated objectives of attaining at least one level higher on Scale A was reached by more 
than 60% who pre-tested at level D (83% attained C or higher) and by more than 80% who pre- 
tested at level E (99% attained D or higher). (See Table 1 9.)* Only 86% of those who pre- 
tested at level F attained level E or higher (100% was the projected figure in the program objec- 
tives) and only 59% who pre-tested at level C attained level B or higher (75% projected) with 3% 
actually being rated lower (level D). It will be shown later in this report that this failure at the C 
level may reflect nothing more than poor initial placement: the major problem with using sub- 
jective teacher evaluation for rating student ability to speak English. What is needed is an instru- 
ment that can be graded uniformly. This will be discussed in detail in the recommendations section 
of this report. 

The Linguistic Capacity Index. The Evaluators decided to use the Linguistic Capacity Index 
because it had been used for pre-test purposes in the fall prior to their receiving the contract. 
Furthermore, it would provide comparative data to the Puerto Rican Scale ratings. The test 
seems much better suited to elementary school youngsters and questions about the administration 
of the test and about several of the items were raised by the evaluation team. (See Appendix M.) 
Nevertheless, the test, which is divided into three sections with twenty questions in each on 
"Vocabulary, Contrastive Phonology and Contrastive Grammar," though designed for use with 



* The same information presented in terms of frequencies can be found in Appendix L. 
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Table 19 

RESULTS OF PUERTO RICAN SCALE A 



Expressed as Percentages 



Pretest 


Predicted 


Actual 




Post Test Distribution 




Level 


Result 


Result 


_£.. 


E 


D_ 


C 


B_ 


A_ 


C 


.75 B or greater 


.59 B or greater 


.00 


.00 


.03 


.39 


.58 


.01 


D 


.60 C or greater 


.83 C or greater 


.00 


.00 


.17 


.72 


.11 


.0u 


E 


.80 Dor greater 


.99 D or greater 


.00 


.01 


.15 


.63 


.19 


.03 


F 


.100 E or greater 


.86 E or greater 


.14 


.45 


.37 


.04 


.00 


.00 



Table 20 

SUMMARY OF LC! RESULTS FOR ESL STUDENTS 



N-332 



Pretest 
Mean 

Vocabulary 16.57 

Contrastive 
Phonology 11.93 

Contrastive 
G rammer 15.52 

Total Score 43.97 
All t ratios are significant at 



Posttest 

S.D. Mean S.D. 

3.62 18.20 2.30 

3.00 13.72 2.89 

3.48 17.09 2.50 

8.82 49.12 6.29 

e .005 level. 



Mean T 

Gain Ratio 

1.63 10.11 

1.79 10.70 

1.57 8.82 

5.15 13.72 



Spanish-speaking learners of ESL reflects many of the learning problems of the BSL learner in 
general. 

The t-ratios for the pre-post-test mean gains were significant at the .005 level for each of 
the individual sections and for the test as a whole. (Table 20) 

A Comparison: Puerto Rican Scale "A "and Linguistic Capacity Index. An examination of student 
pre-test scores on the Linguistic Capacity Index indicated that there was considerable overlap 
between adjacent rating levels on the Puerto Rican Scale in terms of how students in each category 
(pre-test: C to F) scored on the LCI. This can be seen in Table 21 * which indicates that five 
students (in a sample of 289) placed in the E group on the rating scale scored lower than the 
lowest student in the F category on the LCI. A clearer picture of these overlapping scores above 
and below the means for each category is presented in Table 22. Since the range for both groups 
(E: 13 to 55) (F: 26 to 48) were quite wide, there is reason to question the validity of the Puerto 



* The data appear in another format in Appendix N, which includes medians, standard deviations, and quartile ranges. 

23 



9 

ERIC 



< 

LU 
-J 

< 
CJ 
00 

cc 

Ql 

> 
00 

CO 
LU 
CC 

o 
a 

CO 

-J 

< 

H 
O 
H 

a 

S uj g 

0) CC CM 

S ii 
HQ z 



D 

CD 

CC 
H 
CO 

Q 

UJ 



u. 
O 

CO 
CO 

> 

-J 

< 
< 



a 



4\ 



s 



S 



in 

s 



m mm 
is mm 

♦-cm *- en *- co *r cm 



IS iiisll Iil8llgi§l8li888 



1 MM3MKM<MS^HMMS 

i sssissslssigiliiiiiillll 



5555 5 S5l5S5§55S855§SSSSS8^ §5 S 



188 



88 88 8 

^ in io i oq oj 

88 88 8 



§ § 



CM CM 



§§55 i HH8MsS8^2ii|K<MHMRM»H'Stl§ 
88§i l MHMMHHHiHMHgHMHiHM 



pi-r».o»-Oi 



ro o — cm ro tj- m ( 



r^ooc^o^cM<n^in<Dr^ooo>o»-CM<n^in<Dr^ooo>O^CMfOTf-in<Dr«*CJO 
n * m 



9 

ERIC 



24 



Table 22 

A COMPARISON OF THE DISTRIBUTION OF STUDENTS' TOTAL SCORES ON THE 
PRE LCI AND STUDENT PLACEMENT ON THE PRE PUERTO RICAN SCALE A 
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Rican Scale "A" as a placement instrument. This doubt about its validity was supported by the 
teachers' responses to the item concerning homogeneous vs. heterogeneous grouping (see Table 3, 
page 8) and the discussion of the Selection of Student Population preceeding. 



Table 23 

INTERCORR ELATION MATRIX OF PRE SCORES ON PUERTO RICAN 
SCALE A AND PRE SCORES ON LINGUISTIC CAPACITY INDEX* 



Contra stive Contrastive 
Puerto Rican A Vocabulary Phonology Grammar TOTAL 

P.R.A. 1.0000 .5174 .3897 .4300 .5230 

Vocabulary .5174 1.0000 .5688 .6638 .8688 
Contrastive 

Phonology .3897 .5688 1.0000 . 5450 .7976 
Contrastive 

Grammar .4300 .6638 .5450' 1.0000 .8555 

TOTAL .5230 .8688 .7976 .8555 1.0000 

*Signif icant of the .05 level. 

The correlation between the scores of the Puerto Rican Scale "A" and the pre-test totals of 
the LCI was .5230 (See Table 23), which is very low for two presumably comparable instruments. 
Furthermore, the great degree of overlap between categories in the Scale and the total scores on 
the LCI is a further reflection of the inability of each instrument to distinguish between the 
ability levels of students except in the broadest way. 
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SUMMARY AND CONCLUSIONS 

The summary and conclusions are arranged in the same order as the presentation of this 
report. The failure of the data to entirely support attainment of any of the evaluation objectives 
described herein should be considered more of a failure in the proposing of realistic objectives 
than as a failure of the program. 

Student Selection and Placement 

Not stated in terms of a specific program objective in the proposal or research design, student 
selection and placement has been treated in this report since it is part of the basic foundation upon 
which much of the study rests. Although students were selected for the program if they were 
rated between levels C and F on the Puerto Rican Scale "A," or upon recommendation of a teacher or 
guidance counsellor, 89% of the teachers stated that student placement was based on English 
language proficiency. Yet, 69% of the teachers indicated that such proficiency was determined by 
an interview alone, without reference to the Puerto Rican Scale or the Linguistic Capacity Index 
(LCI), instruments which have been used in the City schools for several years. The subjective 
nature of placement through interviews was negatively reflected in that 14% of the teachers re- 
sponded that all of their classes were heterogeneously grouped in terms of students' English pro- 
ficiency and an additional 45% indicated some of their classes were homogeneous and some 
heterogeneous. Only 36% indicated that all of their classes were homogeneous. Furthermore, 
scores on the LCI pre-test suggest that there was considerable overlap in scores among students 
across the Scale levels from C to F. 

The Teaching Staff 

Once again, there was no stated objective concerning the ESL teachers participating in the 
program; however, the evaluators believed that a look at the experience the teachers bring with 
them to the project was a critical aspect of the investigation. More than 50% of the teachers in 
the program have six or more years of teaching experience, while less than 20% have six or more 
years of ESL teaching experience. More than 67% of the teachers have had what are considered 
the fundamental professional courses required for ESL teachers, but less than 40% are members 
of the national professional organization, TESOL (Teachers of English to Speakers of Other 
Languages), and even fewer are members of the state affiliate (15%). These percentages are con- 
sistently lower with the less-experienced teachers in terms of number of years of teaching. 

Almost 60% of the teachers in the program hold ESL licenses, with 33% holding an addi- 
tional license or licenses in speech, English, and/or a foreign language, 

2? 

O 
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In-Service Training 

Rating of Teacher-Trainer Consultants. The stated objective that 90% of the teachers would 
rate the consultants good to excellent was not achieved for any of the items prepared by the 
evaluation team, ranging across such areas as the consultants' knowledge of the field; the number, 
quality and sequencing of workshops and demonstration lessons; and the type, availability and 
quality of the texts, instructional materials, and audio-visual aids provided by the Board. An 
examination of how less-and more- experienced teachers responded to the seventeen different 
items surveyed did not indicate any trends of difference in response. Despite the fact that the 
program objective was not met, teacher response was consistently favorable with the exception of 
their responses to the number and sequencing of both workshops and demonstration lessons, 
which were decidedly negative. This problem is discussed further in the next section, wherein the 
evaluators recommend rejecting the data concerning workshops and demonstration lessons as 
invalid. 

Number of Visits, Workshops and Demonstration Lessons. The stated objective of a monthly 
average of 16 visitations, 2 workshops and 5 demonstration lessons was met only by the average 
number of visitations per month ( 19.54). However, the evaluators received the impression that 
since the distinctions among visitations, workshops, and demonstration lessons had not been 
specifically defined, both the teacher-trainer consultants themselves and the teachers, when res- 
ponding to questions conerning the number of each of these activities, were unclear as to what 
constituted each. Furthermore, the evaluation director also had the impression that the workshops 
held at the Board were expressly for the department chairmen and not for the classroom teachers. 
As a result, the evaluators recommend that all data reflecting numbers, sequencing and quality of 
both workshops and demonstration lessons be ignored as invalid. 

Nevertheless, it was clear from the teachers, who indicated the number of visitations, work- 
shops and demonstration lessons, as well as from the consultants themselves, that the teacher- 
trainer consultants were out in the field in the schools providing the services for which they were 
employed. It should also be mentioned that video-taped teacher training films were made at the 
institutes. 

Awareness of Special Needs of ESL Students 

The objective stated that 90% of the teachers would indicate that the program had made 
them more aware of the special needs of the ESL learner. Although teacher response was on the 
whole positive (at least 50% answered to all but one of the questions), no item received the 
90% affirmative response required to meet the stated objective. It was unwise to assume that 90% 
of any professional group would admit that they had that much to learn about their own field. 
Further examination of the data indicated that there was no differential response among teachers 
with less than two, two to five, or six or more years of ESL teaching experience. A more realistic 
objective of 60% would have permitted nine of the fifteen items surveyed to have obtained that 
objective. 
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Articulation 

Only 35% of the department chairmen surveyed thought that there was articulation between 
their high schools and the feeder junior high schools. Several chairmen indicated that since a 
majority of their students were new arrivals and had not attended a feeder junior high school, the 
question was irrelevant. Others suggested that all efforts to establish such articulation were 
initiated by the high schools but without success. These rationales were sufficient to overcome 
the fact that the 90% affirmative response in the program objective was not achieved. 

The chairmen responded much more favorably (74%) to whether or not there was articulation 
among the various high schools in terms of similarity of program. Several chairmen gave strong 
praise to the efforts of the Central Board staff for providing the leadership that had helped establish 
such articulation, including the dissemination of materials prepared at individual schools and shared 
through the efforts of the Board. These teacher-prepared materials were credited by several chair- 
men as the strength of their programs, individually, and perhaps this resulted in fewer affirmative 
responses to the question of inter-high school program similarity. 

Corroborating the chairmen's responses v/eie responses to two questions concerning the 
number of periods ESL teachers met their ciasses weekly and the textbooks they were using. 
Eighty-three percent of the teachers met their classes 5 to 10 times per week and only 9% of the 
teachers were not using any of four types of materials distributed by the Board. 

Department chairmen (86%) believed that there was good to excellent opportunity for input 
to the Central Board while only 39% of the teachers rated such opportunity good to excellent. 

Effects of Program on Students 

Drop Out Rate. The combined average ESL drop out rate was in fact 50% better than the 
combined total dropout rate for the responding schools, thereby meeting the stated objective. It 
was only through thr* extraordinary efforts of individual department chairmen that data on drop 
out rates became available to the evaluators. 

Puerto Rican Scale A. The stated objectives of attaining at least one level higher on Scale A 
(speaking ability) was reached by the appropriately projected percentages for students who pre- 
tested into levels D and E. Of those who pre-tested in at level F % 86% were rated at least one level 
higher but this figure failed to reach the rather high, projected figure of 100%. Only 59% of those 
who pre-tested at level C attained level B or higher failing to reach the projected figure of 75%. 
Information obtained by using the Linguistic Capacity Index suggests that the major fault of any 
failure to reach the stated objective on the Puerto Rican Scale was probably caused by poor initial 
placement. (See comparison of Puerto Rican Scale and Linguistic Capacity Index below.) 

The Linguistic Capacity Index. Although this test seems better suited to elementary school 
youngsters, it was used by the evaluators for two reasons: pre-test data were available when the 
evaluators received the contract in late February, and it would provide a comparable measure to 
the Puerto Rican Scale ratings. (A separate examination of the LCI by the evaluation team can be 
found in Appendix N.) The t-ratios for the pre-post-test mean gains were significant at the .005 
level for each of the individual components and for the test as a whole. 
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Puerto Rican Scale A and the Linguistic Capacity Index. The LCI scores were set up in groups 
equivalent to those students rated in each of the levels (C to F) on the Scale. What was immediately 
clear was that despite means on the LCI for each group ascending from F through C, there was 
considerable overlap among scores on the LCI for students in each of the Scale categories. This 
suggests that the program did not have homogeneously grouped classes for the most part because 
of a dependence on a rating system that does not permit uniform ratings because it is too depen- 
dent on teacher judgment. 

For two presumably comparable instruments, the correlation coefficient (r) was a rather 
low .5230. 
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RECOMMENDATIONS 



In view of the evaluators' belief that any failure was more in the proposing of realistic 
objectives than in the program itself, the evaluators recommend that the program be continued. 
They further recommend: 

1 . that all terms be clearly defined in any subsequent proposals, including awareness, 
articulation, workshop, demonstration lesson. 

2. that more realistic statistical goals be stated for future proposals, taking into account the 
diversity of human populations involved: students, teachers, department chairmen, and teacher- 
trainer consultants. 

3. that new instruments be sought or created for the purpose of student selection and 
homogeneous placement. Even the Linguistic Capacity Index would be a better placement index 
than the Puerto Rican Scale because there are quantifiable results to be examined, although this 

is not to be construed as a claim that the LCI. is a suitable measure of English language proficiency. 

4. that unappointed teachers licensed by the Board of Examiners to teach ESL be hired 
whenever there is an opening in the program. There is no reason that more than 40% of the teachers 
do not have ESL licenses. At the same time, the Board should be credited for attempting to provide 
in-service training for those who do not hold licenses and even for those who do. Learning does not 
stop when one gets a license. 

5. that, if workshops and demonstrations lessons are to be included in subsequent proposals, 
they be scheduled in schools and at the Board at times convenient for the maximum number of 
personnel to benefit from them, 

6. that, if articulation between the feeder junior high schools and the high schools is solely 
a function of these participating schools, then it should not be listed as a function of the Central 
Board and should thus be omitted from program objectives. 

7. that the efforts to disseminate throughout the schools information about, and newly 
published materials in, the field of English as a second language be continued and intensified. 

8. that, if information on students who leave the program for the mainstream is sought for 
follow-up purposes, procedures should be established so that such information is readily available 
either directly from the schools or from the Central Board. Such information would include current 
ratings on the Puerto Rican Scale (as long as it is used), general academic achievement, attendance 
and behavior. 

9. that, if follow-ups are to be done on drop out rates, once again procedures must be estab- 
lished so that such data are readily available, both for current year students and for those students 
who might have moved into the mainstream after having been "promoted" from the program in 
the current or previous years. 

10. that instruments be created for testing student achievement in all language areas and 
skills taught in the program. No test is currently used for this purpose uniformly throughout the 
program to determine whether students have learned what they studied or not. 
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1 1 . The eyaluators deem it essential that the Central staff foster interest in and encourage 
participation in appropriate professional organizations at the national (TESOL) and local levels 
(NYSESOLBEA). 

12. It was noted in the proposal that each ESL youngster in the program wa'i to have a daily 
period of ESL in addition to his daily regular English class. Consideration should be given to 
providing two periods of ESL daily (this is already being done in some schools), especially to 
those youngsters who are of low English proficiency. A regular English class can do little but 
frustrate such youngsters, who could benefit considerably by the double period or two separate 
periods of ESL. 
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Appendix A 



Last Name First Name School 



School Address 



Dear Teacher: 

Attached you will find a questionnaire consisting of questions designed to elicit various kinds of 
information about you, your classroom activities, and the Board Consultants and program. This 
questionnaire was prepared for use as part of an evaluation of ESL in the New York City high schools. 

We would sincerely appreciate your cooperation in taking the time to respond to the questionnaire 
as carefully and completely as possible. Since we are interested in the total picture rather than each 
of you as individuals, we ask that you do not put your name on the questionnaire, but instead, 
put your name, school with address in the spaces provided at the top of this sheet. 

Please be sure to return this sheet with the completed questionnaire in the return envelope 
provided. It is vital for re-cycling of this program and maintenance of teaching positions that this 
questionnaire be returned no later than Friday, May 29, 1973. 

Thank you f*>: your cooperation in this endeavor. 

Sincerely yours, 

Harvey Nadler 

Director, Evaluation Team 

"Improving the Teaching of ESL" 

S.U.E. Evaluation Team - New York University Center for Educational Research 
Harvey Nadler, Director 



34 



*E = excellent 
G = good 
S = satisfactory 
F = fair 
P = poor 

Consultants P,ease check a PP r °P nate box. 
*E G S F F» 

1. The opportunity to meet with consultants has been: 

2. The number of visits by consultants has been: 

3. The availability of consultants when called has been: 

4. The working relationship with consultants has been: 

5. The quality and expertise of the consultants has been: 



Workshops 

6. The number of workshops has been: 

7. The quality of the workshops has been: 

8. The sequencing of the workshops has been: 



Demonstration Lessons 
9. The number of demonstration lessons has been: 

10. The quality of the demonstration lessons has been: 

11. The sequencing of the demonstration lessons has been: 



Instructional Materials and Textbooks 

12. The variety of instructional materials and textbooks has been: 

13. The availability of instructional materials and textbooks has been: 

14. The quality of instructional materials and textbooks has been: 



"Improving the Teaching of ESL" 

S.U.E. Evaluation Team — New York University Center for Educational Research 
Harvey Nadler, Director 

Audio-Visual Aids 

15. The vari ety of audio-visual aids has been: 

16. The availability of audio-visual aids has been: 

17. The quality of audio-visual aids has been: 

Miscellaneous 

18. How would you rate the clerical demands by the project on 
your time? 
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19. How would you rate opportunity for teacher (YOU) input to 
the Central Board? 



For questions 20 through 23, please indicate the number of each in the space 
to the right of the question. 

Number 

20. How many workshops were provided in your school? 

21. How many workshops did you attend? 

22. How many demonstration lessons were provided in your school? 

23. How many demonstration lessons did you attend? 



The ESL Project sponsored by the Board of Education of the City of New York increased my knowledge of the 
following. 

Please check appropriate box. 
Yes No 

24. The effect of cultural differences on language 

25. The nature of cultural differences 

26. The cultural background of the students you are teaching 

27. The nature of the English grammatical system 

28. The difference between the sound system of English and other 
languages 

29. The implications of linguistics and psychology for the ESL teacher 

30. New materials especially for ESL learners 

31. New approaches to teaching grammar 

32. New approaches to teaching reading 

33. New approaches to teaching composition 

34. New approaches to teaching vocabulary development 

35. New approaches to teaching pronunciation 

36. New approaches to teaching spelling 

37. New approaches to using audio-visual aids in the classroom 

38. New approaches to using field experiences as learning experiences 

Speech, English, 
& Foreign 

39. What NYC license(s) do you hold? TESOL Language Other 
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40. How long have you been teaching? 

41. How long have you been teaching ESL? 



Less 

than 6 years 

2 years 2-5 years or more 



Less More than 

than 5 5 to 10 10 



42. How many periods do you see each ESL class weekly? 

43. When ESL students graduate from your school, do 
you think that their English language proficiency 
would permit them to compete with native English 
speakers? 

For jobs 

At college 

In a training position 

44. Do you have any personnel to assist you in the ESL 
classroom? 

45. How many non-ESL class-periods do you teach each 
week? 

46. Which of the following do you use? 

New Lado Series 

American Book Company Charts 

English 900 

English for today 



Yes 



Yes 



No 



I do not know 



1-5 



Yes 



No 



6- 10 



No 



47. Were you consulted on the selection of textbooks? 



Yes 



No 



48. How many periods do you teach ESL to each ESL 
class each week? 



Less than 5 5-9 10 or more 



Age 



49a. How are students placed in your ESL classes? 



Previous 
Schooling 



English 

Language 

Proficiency 
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If you answered English language proficiency, how 
is this determined? 



Lin- 
guistic 
P.R. Capacity Inter- 
Scale I ndex view Other 



All 

homo- Some homo All hetero- 
geneous Some hetero geneous 

50. What kind of classes do you have, according to their 

English language proficiency? 

Years English 
in Language Academic 

Age School Ability Progress 

51. On what basis are ESL students promoted in your 

school? 

52. Please indicate whether you have had the 
following courses at an accredited college 

or university. Yes No 

a. ESL methodology 

b. ESL theory and/or materials 

c. Phonology and/or phonetics 

d. Contrastive linguistics 

e. General linguistics 

f. Structure of English 

g. Transformational Generative Grammar 



Yes No 

53. Are you a member of National TESOL? 

54. Are you a member of N.Y.S. ESOL BE A? 

55. Did you attend the TESOL Convention in 

a. New York City 

b. Miami 

c. Chicago 

d. San Antonio 

e. New Orleans 

f. Washington, D.C. 

g. San Juan, Puerto Rico 
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56. Did you attend the N.Y.S. ESOL BEA Meeting in 

a. Harlem 

b. Teacher's College 

c. Grossinger's 

d. N.Y.U. (in conjunction with LA.R.C.) 
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Appendix B 

BUREAU OF EDUCATIONAL RESEARCH 
Board of Education of the City of New York 
1 1 0 Livingston Street, Brooklyn, New York 1 1 202 



Account of Activities of Teacher Trainers in the High School 
English as a Second Language Program 



For Month of , 19 



Name of Teacher Trainer 

Total Number of Visits Made to Schools: 
Total Number of Workshops Conducted: 
Total Number of Demonstration Lessons Given: 
Total Number of Other Activities Conducted: 
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Appendix C 

TO: Department Chairmen 

FROM: Harvey Nadler, Director, Evaluation Team 

RE: SALE. Evaluation: "Improving the Teaching of ESL/' 

As part of the evaluation of the S.U.E. Program "Improving the Teaching of ESL" we have been 
asked to gather data concerning the coordination, resulting from the program sponsored by the 
Central Board, between your program and the junior high schools and the similarity between yours 
and the other high schools participating in the ESL program. 

We would appreciate your taking the time to check and comment (if you like) on the two questions 
which appear on the attached sheet. Please return not later than June 4, 1973. 



Thank you for your cooperation in this matter. 
HN:pc 

Evaluation: "Improving the Teaching of ESL" 



Very truly yours. 



Harvey Nadler 

Director, Evaluation Team 



A, As regards the ESL learners, is there successful coordination between your program and the 
junior high schools from which they have come? 

yes no I do not know 

Comments: 



B, Is there similarity between your ESL program and the ESL programs of the other high schools 
participating in the project? 

yes no • ' j I do not know 

Comments: 
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Appendix D 
S.U.E. EVALUATION TEAM 
New York University Center for Educational Research 
Drop Out Data 



School 



First Term Second Term 

1. School Drop Out Rate 

2. Number of School Drop Outs 

3. Total ESL Drop Out Rate 

4. Total Number of ESL Drop Outs 

5. ESL Drop Out Rate - City Funded 

6. Number of ESL Drop Outs - City Funded 

7. ESL Drop Out Rate - Project Funded 

8. Number of ESL Drop Outs — Project Funded 



First Term 
City Project Regular 
Funded Funded School 
ESL ESL Program 

School Wide 

Beginning Registration 

less Transfers 

Net Registration 

less Ending Registration 

Registration Difference ^ === ^^ = ^ == ^— = 

Eleventh Grade * 

Beginning Registration 

less Transfers 

Net Registration 

less Ending Registration 

Registration Difference 



'For the eleventh grade the figures should be for those students that have participated in the ESL program and are now in the 
main stream and other students in the main stream. 



Second Term 
City Project Regular 
Funded Funded School 
ESL ESL Program 
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Appendix K 
S.U.E. EVALUATION TEAM 



ERIC 



New York University Center for Educational Research 
High School Chairmen Data 



School Chairman 



1. How many workshops were provided in your school? 

2. How many demonstration lessons were provided in your school? 

3. How many visits did consultants from the Board of Education make to 
your school? 

4. How many teachers received time to go to the Board of Education for 
meetings? 

5. How many project funded ESL teachers are in your school? 

6. How many city funded ESL teachers are in your school? 

7. How would you rate your opportunity for input to the Central Board? 
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Appendix L 
RESULTS OF PUERTO RICAN SCALE A 



Expressed as Frequencies 



Post Test Distribution 



Pretest Level 


Total 


_F 


E_ 


D 


C_ 


B_ 


A_ 


C 


192 


0 


0 


5 


74 


111 


2 


D 


208 


0 


1 


35 


150 


22 


0 


E 


142 


1 


21 


89 


27 
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142 


20 


64 


52 


6 
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0 
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ERIC 



Appendix M 



The Linguistic Capacity Index test utilized in this study has several problems for the testee 
both in the instruction which he is expected to follow and in the items to which he must respond. 

The instruction for the pre-test badly mislead the testees by suggesting that they will have to 
match up a picture at the right with the one at the extreme left, whereas the picture at the extreme 
left is merely a guide for those testees who cannot read (recognize, understand) the number for 
each item. Other problems with instructions are discussed under specific items below. 

A . Vocabulary Recognition 

#5 The interpretation of the meaning of the word with determines whether the testees are 
marked correct since there are two pictures in which there are a boy and a book, but only one in 
which the boy is holding the book (in the other he is apparently reaching for it), the former being 
the expected correct response. 

#8 The correct picture contains a boy seemingly walking up a road to a schoolhouse and the 
testee is instructed to "Mark the boy who is going to school." Another picture, however, contains 
a boy sitting at what is obviously a school desk. This boy is /;/ school and NOT going to school, but 
if a testee in his own mind misconstrues the tenses ("going to school" and "goes to school") then 
he could select the other picture, and be marked wrong on what is in fact a grammatical distinction 
and not one of vocabulary recognition. 

#10 instructs the testee to "Mark the boy who is going across (sic)." Without an object this is 
an odd construction. The picture indicates a boy crossing a bridge. If the testers were testing vocab- 
ulary recognition and not function words like across, they might have asked the testee to identify 
the word bridge. However, even testing across could have been handled better by showing a boy 
going across, going under, and (perhaps magically) going (flying) over a bridge. (The other two 
choices in the given item show a boy standing on a box and a boy just standing.) 

#12 This item suddenly forces the testee to shift the focus for the item contains (from left to 
right) 1 . a boy looking to the right 2. a boy looking under a house, and 3. another boy looking to 
the right. The instructions are to "Mark the boy who is looking at the house." Unlike previous items 
wherein each of the three pictures was a discrete item, here the testee is faced with a total picture 
consisting of the three items listed above. (In addition to the shift of emphasis, the testers are once 
again testing function words (as in items 5, 8, 9, 10, and 1 1); perhaps, it is a matter of orientation 
(the evaluators') but vocabulary recognition suggests that those being tested will be able to recog- 
nize concrete objects or concepts.) 

#1 3 Instructions are to "Mark the boy who is going down." (a sliding pond)-a recurrence of 
the problem described in #10. The other two boys in the picture are sitting at a school desk (as in 
#8) and standing (as in #10). 

#16 through #20 These items create double confusion, since the instructions change to "Draw 
a circle around . . ." (the students, however, have been alerted to this in the general directions: 
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'Trior to the administration of the test the teacher must instruct the pupils in marking a line 
through a picture and drawing a circle." (Manual of Instructions, p. 2) However, the testees in items 
16 through 20 are told, respectively, to draw a circle around 

one of three boys in *1 6 (a boy) 

two (3, 4, 5?) of the five dogs in #17 (some dogs) 

all five apples in #18 (all the apples)-the best any testee could do is an elliptical figure at any 

rate. 

around the two cats in #19 (both the cats), and 

in #20, the testee must draw five circles (!), one around each of the five dogs illustrated (each 

dog). 

One might also ask why all those pictures that contain a child or children in the first section 
(Vocabulary Recognition) consist only of pictures of boys. A girl appears for the first time in the 
fourth item in the phonology section. 

B. Contrast ire Phonology 

#5 Instructions are to "Mark pull," with pictures of a boy pulling a wagon, a boy apparently 
holding his nose as he goes off a diving board into a pool, and a flagpofc with an American flag. 
Even the least purist among educators would be taken aback by the grammar (meaning) of the in- 
structions to the testee. What about the non-native learner? Perhaps, he gets it correct, but is he 
also learning that this is correct English? (In #15 with the same three pictures, the instructions are 
to "Mark the pool.") 

#7 The instructions are for the testee to "Mark the cupboard." How many non-native young- 
sters have heard the word or why they should is beside the point-will they recognize the middle 
picture of the cupboard and not the first picture of a boy in bed, or the third of four cups? Is this 
contrastive phonology? 

#10 Instructions are to "Mark the leather," with pictures of a ladder, a letter, and the sole of 
a shoe. 

#1 1 Instructions are to "Mark the sink," with pictures of a boy sitting (and thinking?), a 
woman with her hands in the sink, and a woman, mouth agape, holding a book, with musical notes 
off to the left (singing). Apparently, if the authors had chosen either o^ the other two choices, they 
would have suggested that testees "Mark think (the think?)" or "Mark sing (the sing?)"-in fact, 
"Mark the sing." turn up as the instructions for #20. 

#17 consists of pictures of a shoe, a boy chewing (with a bite seen missing from the sandwich 
he has in his hand) and the same girl who appeared in #4 and #8 (but whose activity is unclear to 
the cvaluators). To be consistent (see #5) if the second picture is the correct answer, the instruc- 
tions should read "Mark chew," but they say, "Mark the chew." 

C. Contrastive Grammar 

#1 The inclusion of the middle picture of a dog is meaningless in an item that calls for the 



testee to "Mark the bird that can fly away." (italics added) 

#6 Similarly, the third picture of a boy alone, in an item that calls for the testee to "Mark the 
boy who likes the dog." is meaningless. 

• * '#9 contains a pjcture v of a bird with beak open and musical notes coming out, a dog, and an- 
* other bird with beak closed. The instructions are to "Mark the animal that doesn't sing." A bright 
child might mark the dog as soon as he heard the word animal; nevertheless, some birds do not sing 
and the third picture may imply an ambiguous correct choice. 

#10 contains a picture of two boys (one tall in long pants, one short in short pants) and a 
girl. The instructions are to "Mark the boy who is taller." 

#12 contains the same three pictures as in #3, with the same instructions except that #3 says, 
"Mark the dog that has eaten." and #12, "Mark the animal that has eaten." (italics added) 

#8 and #18 are "Mark the boy who is cold." and ". . . who has a cold," respectively with 
pictures of a boy in bed apparently with a thermometer in his mouth, and a table with medicine nearby, 
and a boy dressed warmly (including ear laps), a snowman in the background; the boy is apparently 
shivering as he clutches his elbows with his hands, his knees bent with extra lines drawn to indicate 
the shivering. A boy who is cold can have a cold and vice versa; true the grammatical difference 
does exist, but is this item worth testing? 

The major question about the grammatical contrasts, however, is based on the fact that the in- 
structions for each item contain a relative clause ("Mark the X who or that . . .") with the excep- 
tion of #15 ("Mark the doghouse.") Admittedly, there is consistency in the use of these relative 
clauses, but some of the items might have been simplified (at least in "surface" terms of the number 
of words that the testees had to listen to and understand) e.g. #4 the jumping boy; #8 the cold boy; 
#10 the taller boy; #17 the painted rabbit; and #19 the tallest boy. Perhaps using both types woUI?^ 
not have permitted the testers to contrast the items. 
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